Optimum Follow the Leader Algorithm

Consider the following setting for an on-line algorithm (introduced in [FS97]) that learns from a set of experts: In trial t the algorithm chooses an expert with probability pi. At the end of the trial a loss vector 1 L ∈ [0, R] for the n experts is received and an expected loss of ∑ i p t iL t i is incurred. A simple algorithm for this setting is the Hedge algorithm which uses the probabilities pi ∼ exp−ηL <t i . This algorithm and its analysis is a simple reformulation of the randomized version of the Weighted Majority algorithm (WMR) [LW94] which was designed for the absolute loss. The total expected loss of the algorithm is close to the total loss of the best expert L∗ = mini L ≤T i . That is, when the learning rate is optimally tuned based on L∗, R and n, then the total expected loss of the Hedge/WMR algorithm is at most